Determining an impact of a proposed dialog act using model-based textual analysis

ABSTRACT

A computer-implemented method according to one embodiment includes receiving, utilizing a processor, textual data associated with a real-time conversation between a first participant and a second participant, receiving, utilizing the processor, an objective of the first participant for the real-time conversation between the first participant and the second participant, receiving, utilizing the processor, a proposed dialog act to be entered at a current point in the real-time conversation from the first participant, determining, utilizing the processor, an impact of the proposed dialog act on the objective of the first participant, utilizing a model, and performing, utilizing the processor, one or more actions based on the impact of the proposed dialog act.

BACKGROUND

The present invention relates to textual data analysis, and morespecifically, this invention relates to analyzing proposed textualcommunications and determining an impact of those textualcommunications.

Communication between two or more participants is an integral part ofmodern society. For example, many entities rely on effectivecommunication between their representatives and other users in order toresolve issues. However, current prediction analytics do not implementmeasures to optimize a probability of a particular outcome occurring fora participant in the conversation.

SUMMARY

A computer-implemented method according to one embodiment includesreceiving, utilizing a processor, textual data associated with areal-time conversation between a first participant and a secondparticipant, receiving, utilizing the processor, an objective of thefirst participant for the real-time conversation between the firstparticipant and the second participant, receiving, utilizing theprocessor, a proposed dialog act to be entered at a current point in thereal-time conversation from the first participant, determining,utilizing the processor, an impact of the proposed dialog act on theobjective of the first participant, utilizing a model, and performing,utilizing the processor, one or more actions based on the impact of theproposed dialog act.

Other aspects and embodiments of the present invention will becomeapparent from the following detailed description, which, when taken inconjunction with the drawings, illustrate by way of example theprinciples of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a cloud computing node according to an embodiment of thepresent invention.

FIG. 2 depicts a cloud computing environment according to an embodimentof the present invention.

FIG. 3 depicts abstraction model layers according to an embodiment ofthe present invention.

FIG. 4 illustrates a method for recommending a dialog act to aparticipant in a conversation, in accordance with one embodiment.

FIG. 5 illustrates a method for determining an impact of a proposeddialog act in a conversation, in accordance with one embodiment.

FIG. 6 illustrates a method for training a model to determine dialogacts for a conversation, in accordance with one embodiment.

FIG. 7 illustrates a method for training a model to score proposeddialog acts for a conversation, in accordance with one embodiment.

FIG. 8 illustrates an exemplary dialog act recommendation system, inaccordance with one embodiment.

FIG. 9 illustrates an exemplary dialog act validation system, inaccordance with one embodiment.

FIG. 10 illustrates an exemplary progression of an MDP over time, inaccordance with one embodiment.

DETAILED DESCRIPTION

The following description discloses several preferred embodiments ofsystems, methods and computer program products for determining an impactof a proposed dialog act using model-based textual analysis. Variousembodiments provide a method to analyze real-time conversation data andanalyze proposed dialog acts in the context of an objective of aparticipant in the communication.

The following description is made for the purpose of illustrating thegeneral principles of the present invention and is not meant to limitthe inventive concepts claimed herein. Further, particular featuresdescribed herein can be used in combination with other describedfeatures in each of the various possible combinations and permutations.

Unless otherwise specifically defined herein, all terms are to be giventheir broadest possible interpretation including meanings implied fromthe specification as well as meanings understood by those skilled in theart and/or as defined in dictionaries, treatises, etc.

It must also be noted that, as used in the specification and theappended claims, the singular forms “a,” “an” and “the” include pluralreferents unless otherwise specified. It will be further understood thatthe terms “includes” and/or “comprising,” when used in thisspecification, specify the presence of stated features, integers, steps,operations, elements, and/or components, but do not preclude thepresence or addition of one or more other features, integers, steps,operations, elements, components, and/or groups thereof.

The following description discloses several preferred embodiments ofsystems, methods and computer program products for determining an impactof a proposed dialog act using model-based textual analysis.

In one general embodiment, a computer-implemented method includesreceiving, utilizing a processor, textual data associated with areal-time conversation between a first participant and a secondparticipant, receiving, utilizing the processor, an objective of thefirst participant for the real-time conversation between the firstparticipant and the second participant, receiving, utilizing theprocessor, a proposed dialog act to be entered at a current point in thereal-time conversation from the first participant, determining,utilizing the processor, an impact of the proposed dialog act on theobjective of the first participant, utilizing a model, and performing,utilizing the processor, one or more actions based on the impact of theproposed dialog act.

In another general embodiment, a computer program product fordetermining an impact of a proposed dialog act in a conversationincludes a computer readable storage medium having program instructionsembodied therewith, wherein the computer readable storage medium is nota transitory signal per se, and where the program instructions areexecutable by a processor to cause the processor to perform a methodcomprising receiving textual data associated with a real-timeconversation between a first participant and a second participant,utilizing the processor, receiving an objective of the first participantfor the real-time conversation between the first participant and thesecond participant, utilizing the processor, receiving a proposed dialogact to be entered at a current point in the real-time conversation fromthe first participant, utilizing the processor, determining, utilizingthe processor, an impact of the proposed dialog act on the objective ofthe first participant, utilizing a model, and performing one or moreactions based on the impact of the proposed dialog act, utilizing theprocessor.

In another general embodiment, a system includes a processor, and logicintegrated with the processor, executable by the processor, orintegrated with and executable by the processor, where the logic isconfigured to receive, utilizing the processor, textual data associatedwith a real-time conversation between a first participant and a secondparticipant, receive, utilizing the processor, an objective of the firstparticipant for the real-time conversation between the first participantand the second participant, receive, utilizing the processor, a proposeddialog act to be entered at a current point in the real-timeconversation from the first participant, determine, utilizing theprocessor, an impact of the proposed dialog act on the objective of thefirst participant, utilizing a model, and perform, utilizing theprocessor, one or more actions based on the impact of the proposeddialog act.

It is understood in advance that although this disclosure includes adetailed description on cloud computing, implementation of the teachingsrecited herein are not limited to a cloud computing environment. Rather,embodiments of the present invention are capable of being implemented inconjunction with any other type of computing environment now known orlater developed.

Cloud computing is a model of service delivery for enabling convenient,on-demand network access to a shared pool of configurable computingresources (e.g. networks, network bandwidth, servers, processing,memory, storage, applications, virtual machines, and services) that canbe rapidly provisioned and released with minimal management effort orinteraction with a provider of the service. This cloud model may includeat least five characteristics, at least three service models, and atleast four deployment models.

Characteristics are as follows:

On-demand self-service: a cloud consumer can unilaterally provisioncomputing capabilities, such as server time and network storage, asneeded automatically without requiring human interaction with theservice's provider.

Broad network access: capabilities are available over a network andaccessed through standard mechanisms that promote use by heterogeneousthin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

Resource pooling: the provider's computing resources are pooled to servemultiple consumers using a multi-tenant model, with different physicaland virtual resources dynamically assigned and reassigned according todemand. There is a sense of location independence in that the consumergenerally has no control or knowledge over the exact location of theprovided resources but may be able to specify location at a higher levelof abstraction (e.g., country, state, or datacenter).

Rapid elasticity: capabilities can be rapidly and elasticallyprovisioned, in some cases automatically, to quickly scale out andrapidly released to quickly scale in. To the consumer, the capabilitiesavailable for provisioning often appear to be unlimited and can bepurchased in any quantity at any time.

Measured service: cloud systems automatically control and optimizeresource use by leveraging a metering capability at some level ofabstraction appropriate to the type of service (e.g., storage,processing, bandwidth, and active user accounts). Resource usage can bemonitored, controlled, and reported providing transparency for both theprovider and consumer of the utilized service.

Service Models are as follows:

Software as a Service (SaaS): the capability provided to the consumer isto use the provider's applications running on a cloud infrastructure.The applications are accessible from various client devices through athin client interface such as a web browser (e.g., web-based e-mail).The consumer does not manage or control the underlying cloudinfrastructure including network, servers, operating systems, storage,or even individual application capabilities, with the possible exceptionof limited user-specific application configuration settings.

Platform as a Service (PaaS): the capability provided to the consumer isto deploy onto the cloud infrastructure consumer-created or acquiredapplications created using programming languages and tools supported bythe provider. The consumer does not manage or control the underlyingcloud infrastructure including networks, servers, operating systems, orstorage, but has control over the deployed applications and possiblyapplication hosting environment configurations.

Infrastructure as a Service (IaaS): the capability provided to theconsumer is to provision processing, storage, networks, and otherfundamental computing resources where the consumer is able to deploy andrun arbitrary software, which can include operating systems andapplications. The consumer does not manage or control the underlyingcloud infrastructure but has control over operating systems, storage,deployed applications, and possibly limited control of select networkingcomponents (e.g., host firewalls).

Deployment Models are as follows:

Private cloud: the cloud infrastructure is operated solely for anorganization. It may be managed by the organization or a third party andmay exist on-premises or off-premises.

Community cloud: the cloud infrastructure is shared by severalorganizations and supports a specific community that has shared concerns(e.g., mission, security requirements, policy, and complianceconsiderations). It may be managed by the organizations or a third partyand may exist on-premises or off-premises.

Public cloud: the cloud infrastructure is made available to the generalpublic or a large industry group and is owned by an organization sellingcloud services.

Hybrid cloud: the cloud infrastructure is a composition of two or moreclouds (private, community, or public) that remain unique entities butare bound together by standardized or proprietary technology thatenables data and application portability (e.g., cloud bursting forload-balancing between clouds).

A cloud computing environment is service oriented with a focus onstatelessness, low coupling, modularity, and semantic interoperability.At the heart of cloud computing is an infrastructure comprising anetwork of interconnected nodes.

Referring now to FIG. 1, a schematic of an example of a cloud computingnode is shown. Cloud computing node 10 is only one example of a suitablecloud computing node and is not intended to suggest any limitation as tothe scope of use or functionality of embodiments of the inventiondescribed herein. Regardless, cloud computing node 10 is capable ofbeing implemented and/or performing any of the functionality set forthhereinabove.

In cloud computing node 10 there is a computer system/server 12, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributed cloudcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed cloud computing environment, program modules may be locatedin both local and remote computer system storage media including memorystorage devices.

As shown in FIG. 1, computer system/server 12 in cloud computing node 10is shown in the form of a general-purpose computing device. Thecomponents of computer system/server 12 may include, but are not limitedto, one or more processors or processing units 16, a system memory 28,and a bus 18 that couples various system components including systemmemory 28 to processor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

Referring now to FIG. 2, illustrative cloud computing environment 50 isdepicted. As shown, cloud computing environment 50 includes one or morecloud computing nodes 10 with which local computing devices used bycloud consumers, such as, for example, personal digital assistant (PDA)or cellular telephone 54A, desktop computer 54B, laptop computer 54C,and/or automobile computer system 54N may communicate. Nodes 10 maycommunicate with one another. They may be grouped (not shown) physicallyor virtually, in one or more networks, such as Private, Community,Public, or Hybrid clouds as described hereinabove, or a combinationthereof. This allows cloud computing environment 50 to offerinfrastructure, platforms and/or software as services for which a cloudconsumer does not need to maintain resources on a local computingdevice. It is understood that the types of computing devices 54A-N shownin FIG. 2 are intended to be illustrative only and that computing nodes10 and cloud computing environment 50 can communicate with any type ofcomputerized device over any type of network and/or network addressableconnection (e.g., using a web browser).

Referring now to FIG. 3, a set of functional abstraction layers providedby cloud computing environment 50 (FIG. 2) is shown. It should beunderstood in advance that the components, layers, and functions shownin FIG. 3 are intended to be illustrative only and embodiments of theinvention are not limited thereto. As depicted, the following layers andcorresponding functions are provided:

Hardware and software layer 60 includes hardware and softwarecomponents. Examples of hardware components include: mainframes 61; RISC(Reduced Instruction Set Computer) architecture based servers 62;servers 63; blade servers 64; storage devices 65; and networks andnetworking components 66. In some embodiments, software componentsinclude network application server software 67 and database software 68.

Virtualization layer 70 provides an abstraction layer from which thefollowing examples of virtual entities may be provided: virtual servers71; virtual storage 72; virtual networks 73, including virtual privatenetworks; virtual applications and operating systems 74; and virtualclients 75.

In one example, management layer 80 may provide the functions describedbelow. Resource provisioning 81 provides dynamic procurement ofcomputing resources and other resources that are utilized to performtasks within the cloud computing environment. Metering and Pricing 82provide cost tracking as resources are utilized within the cloudcomputing environment, and billing or invoicing for consumption of theseresources. In one example, these resources may include applicationsoftware licenses. Security provides identity verification for cloudconsumers and tasks, as well as protection for data and other resources.User portal 83 provides access to the cloud computing environment forconsumers and system administrators. Service level management 84provides cloud computing resource allocation and management such thatrequired service levels are met. Service Level Agreement (SLA) planningand fulfillment 85 provide pre-arrangement for, and procurement of,cloud computing resources for which a future requirement is anticipatedin accordance with an SLA.

Workloads layer 90 provides examples of functionality for which thecloud computing environment may be utilized. Examples of workloads andfunctions which may be provided from this layer include: mapping andnavigation 91; software development and lifecycle management 92; virtualclassroom education delivery 93; data analytics processing 94;transaction processing 95; and card stunt as a service (CaaS) 96.

Now referring to FIG. 4, a flowchart of a method 400 for recommending adialog act to a participant in a conversation is shown according to oneembodiment. The method 400 may be performed in accordance with thepresent invention in any of the environments depicted in FIGS. 1-3 and8-9, among others, in various embodiments. Of course, more or lessoperations than those specifically described in FIG. 4 may be includedin method 400, as would be understood by one of skill in the art uponreading the present descriptions.

Each of the steps of the method 400 may be performed by any suitablecomponent of the operating environment. For example, in variousembodiments, the method 400 may be partially or entirely performed byone or more servers, computers, or some other device having one or moreprocessors therein. The processor, e.g., processing circuit(s), chip(s),and/or module(s) implemented in hardware and/or software, and preferablyhaving at least one hardware component may be utilized in any device toperform one or more steps of the method 400. Illustrative processorsinclude, but are not limited to, a central processing unit (CPU), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), etc., combinations thereof, or any other suitablecomputing device known in the art.

As shown in FIG. 4, method 400 may initiate with operation 402, wheretextual data associated with a real-time conversation between a firstparticipant and a second participant is received. In one embodiment, thetextual data may be created from one or more spoken utterances. Forexample, a speech-to-text application may create the textual data from aspoken conversation between the first participant and the secondparticipant.

Additionally, in one embodiment, the textual data may include aplurality of words, terms, and/or other typewritten information sentbetween the first participant and the second participant during thereal-time conversation. In another embodiment, the real-timeconversation may include a conversation that is occurring in real-timebetween the first participant and the second participant. In yet anotherembodiment, the first participant and the second participant may eachinclude an individual (e.g., a human user, etc.) or an automatedapplication.

Further, in one embodiment, the real-time conversation may include acustomer service conversation. For example, the first participant mayinclude a customer service representative (CSR), and the secondparticipant may include a customer (e.g., a consumer of a good orservice). In another embodiment, the textual data may include aconversation flow between the first participant and the secondparticipant. For example, the conversation flow may include a pluralityof conversation turns. In another example, each conversation turn mayinclude an individual exchange of textual data from the firstparticipant to the second participant, or from the second participant tothe first participant. In yet another example, each conversation turnmay be made in response to a preceding conversation turn.

Further still, in one embodiment, the textual data may be identified byhardware in a computing device. For example, the hardware may includeone or more processors. In another example, the computing device mayinclude a server, a cloud computing environment, etc. In anotherembodiment, the textual data may be received as a result of monitoringthe real-time conversation. In yet another embodiment, the real-timeconversation may include additional participants in addition to thefirst participant and the second participant.

Additionally, as shown in FIG. 4, method 400 may proceed with operation404, where an objective of the first participant for the real-timeconversation between the first participant and the second participant isreceived. In one embodiment, the objective may include textual dataindicating a desired outcome of the real-time conversation for the firstparticipant. In another embodiment, the objective may include amaximization of satisfaction of the second participant with respect tothe real-time conversation, a minimization of frustration by the secondparticipant with respect to the real-time conversation, a resolution ofa problem of the second participant as a result of the real-timeconversation, an answering of a question by the first participant duringthe real-time conversation, etc. In yet another embodiment, theobjective may be selected from one or more pre-determined options (e.g.,by the first participant, etc.).

Also, as shown in FIG. 4, method 400 may proceed with operation 406,where a dialog act to be entered by the first participant at a currentpoint in the real-time conversation that meets the objective isdetermined, utilizing a model. In one embodiment, the dialog act mayinclude a description of an action that is to be performed by thesending of one or more portions of the textual data from one participantto another participant within the real-time conversation. In anotherembodiment, the action may include apologizing, asking a yes/noquestion, providing an informative statement, requesting information,acknowledging a statement made by another participant, etc.

Furthermore, in one embodiment, the current point in the real-timeconversation may include a current turn for the first participant in theconversation. For example, the real-time conversation may include aturn-taking conversation where the first participant and the secondparticipant speak one at a time in alternating turns. In anotherexample, the current point in the real-time conversation may include acurrent opportunity for the first participant to input textual datawithin the real-time conversation (e.g., in response to a previous turntaken by the second participant, etc.).

Further still, in one embodiment, a dialog act may be determined thatmaximizes a probability of the objective of the first participant beingachieved during the current point in the real-time conversation. Inanother embodiment, for each of a plurality of different dialog acts, alikelihood that the dialog act will achieve the objective of the firstparticipant at the current point in the real-time conversation may bedetermined.

For example, each dialog act may be scored, where a lower score mayindicate that the dialog act will be less likely to achieve theobjective of the first participant at the current point in the real-timeconversation when compared to a dialog act with a higher validity score.In another example, a dialog act with a highest score may be selected.

Also, in one embodiment, the dialog act may be analyzed in associationwith earlier points in the real-time conversation (e.g., previous turnstaken within the real-time conversation by the first participant and thesecond participant). In another embodiment, the dialog act may beanalyzed in association with earlier conversations other than thereal-time conversation. In yet another embodiment, the dialog act may beapplied to the model, where the model is trained using earlierconversations.

For example, the earlier conversations may include one or moreconversations of the first participant and the second participant. Inanother example, the model may be trained by identifying one or more ofdialog acts, features, and objectives of the earlier conversations. Forinstance, the features may include one or more of n-grams (e.g.,sequences of one or more words) within the textual data, temporal ortopic based information, tone/emotion, etc. In yet another example, themodel may identify one or more of dialog acts, features, and theobjective of the real-time conversation. In still another example, themodel may include a sequential machine learning model (e.g., a MarkovModel, etc.).

Additionally, as shown in FIG. 4, method 400 may proceed with operation408, where the dialog act is returned to the first participant. In oneembodiment, returning the dialog act may include displaying the dialogact to the first participant. In another embodiment, additional textualdata may be received from the first participant in response to thereturning of the dialog act.

For example, an additional dialog act associated with the additionaltextual data may be determined. In another example, the additionaldialog act may be compared to the returned dialog act. In yet anotherexample, the additional textual data may be conditionally entered, basedon the comparison. For instance, the additional textual data may beentered at the current point in the real-time conversation when it isdetermined that the additional dialog act matches the returned dialogact. In another instance, the additional textual data may not be enteredat the current point in the real-time conversation when it is determinedthat the additional dialog act does not match the returned dialog act.

Further, in one embodiment, a plurality of different objectives of thefirst participant (or other participants different from the firstparticipant) may be received for the real-time conversation between thefirst participant and the second participant, and for each differentobjective, a different dialog act to be entered by the first participantat a current point in the real-time conversation that maximizes aprobability of the objective being achieved during the current point inthe real-time conversation may be determined and returned.

Further still, in one embodiment, a plurality of dialog acts to beentered by the first participant at a current point in the real-timeconversation may be determined that meet the objective, utilizing themodel. In another embodiment, each of the plurality of dialog acts maybe returned to the first participant in an ordered list. For example,for each of the plurality of dialog acts, a probability that the dialogact will meet the objective may be determined, and the list may beordered by this probability.

In this way, objective-oriented dialog acts may be determined, utilizinga constructed analytical model. By offering dialog acts based on priorconversation turns with a goal of achieving an optimal implementation ofa specifically stated objective, this implementation may improve uponearlier prediction implementations, which may only perform basicrecognition and labeling actions/analysis. Additionally, more accuratedialog act suggestions may minimize a necessary amount of turns within aconversation, which may minimize a necessary amount of turns within theconversation, and which may in turn reduce an amount of network trafficbetween the first participant and the second participant as well asprocessing resources necessary to implement such conversation (e.g., ata device of the first and/or second participant, at a network datatransmission device, etc.).

Now referring to FIG. 5, a flowchart of a method 500 for determining animpact of a proposed dialog act in a conversation is shown according toone embodiment. The method 500 may be performed in accordance with thepresent invention in any of the environments depicted in FIGS. 1-3 and8-9, among others, in various embodiments. Of course, more or lessoperations than those specifically described in FIG. 5 may be includedin method 500, as would be understood by one of skill in the art uponreading the present descriptions.

Each of the steps of the method 500 may be performed by any suitablecomponent of the operating environment. For example, in variousembodiments, the method 500 may be partially or entirely performed byone or more servers, computers, or some other device having one or moreprocessors therein. The processor, e.g., processing circuit(s), chip(s),and/or module(s) implemented in hardware and/or software, and preferablyhaving at least one hardware component may be utilized in any device toperform one or more steps of the method 500. Illustrative processorsinclude, but are not limited to, a central processing unit (CPU), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), etc., combinations thereof, or any other suitablecomputing device known in the art.

As shown in FIG. 5, method 500 may initiate with operation 502, wheretextual data associated with a real-time conversation between a firstparticipant and a second participant is received. Additionally, method500 may proceed with operation 504, where an objective of the firstparticipant for the real-time conversation between the first participantand the second participant is received. Further, method 500 may proceedwith operation 506, where a proposed dialog act to be entered at acurrent point in the real-time conversation is received from the firstparticipant.

In one embodiment, the proposed dialog act may include a description ofan action that is to be performed by the sending of one or more portionsof the textual data from one participant to another participant withinthe real-time conversation. In another embodiment, the action mayinclude apologizing, asking a yes/no question, providing an informativestatement, requesting information, acknowledging a statement made byanother participant, etc.

Further still, in one embodiment, the current point in the real-timeconversation may include a current turn for the first participant in theconversation. For example, the real-time conversation may include aturn-taking conversation where the first participant and the secondparticipant speak one at a time in alternating turns. In anotherexample, the current point in the real-time conversation may include acurrent opportunity for the first participant to input textual datawithin the real-time conversation (e.g., in response to a previous turntaken by the second participant, etc.).

Also, in one embodiment, the proposed dialog act may be identified fromtextual input provided by the first participant. For example, thetextual input may be analyzed in order to determine the proposed dialogact that is implemented through the textual input.

In addition, method 500 may proceed with operation 508, where an impactof the proposed dialog act on the objective of the first participant isdetermined, utilizing a model. In one embodiment, determining the impactmay include scoring the proposed dialog act on a predetermined scale.For example, the impact may include a validity score for the proposeddialog act. In another example, the validity score may be on apredetermined scale (e.g., 1-10, etc.).

Furthermore, in one embodiment, determining the impact may includedetermining an amount that the proposed dialog act will change aprobability of the objective of the first participant being achievedduring the current point in the real-time conversation. In anotherembodiment, determining the impact may include determining a likelihoodthat the proposed dialog act will achieve the objective of the firstparticipant at the current point in the real-time conversation. Forexample, a lower validity score may indicate that the proposed dialogact will be less likely to achieve the objective of the firstparticipant at the current point in the real-time conversation whencompared to a proposed dialog act with a higher validity score.

Further still, in one embodiment, determining the impact may includeanalyzing the proposed dialog act in association with earlier points inthe real-time conversation (e.g., previous turns taken within thereal-time conversation by the first participant and the secondparticipant). In another embodiment, determining the impact may includeanalyzing the proposed dialog act in association with earlierconversations other than the real-time conversation.

Also, in one embodiment, determining the impact may include applying theproposed dialog act to the model, where the model is trained usingearlier conversations. For example, earlier conversations may includeone or more of the first participant and the second participant. Inanother example, the model may be trained by identifying dialog acts,features, and objectives of the earlier conversations. In yet anotherexample, the features may include one or more of n-grams (e.g.,sequences of one or more words) within the textual data, temporal ortopic based information, tone/emotion, etc. In still another example,the model may identify dialog acts, features, and the objective of thereal-time conversation. In another example, the model may include asequential machine learning model (e.g., a Markov Model, etc.).

Additionally, method 500 may proceed with operation 510, where one ormore actions are performed, based on the impact of the proposed dialogact. In one embodiment, the one or more actions may include sending anotification including the impact of the proposed dialog act on theobjective of the first participant. For example, the notification may besent to the first participant, an administrator, etc. In anotherembodiment, the one or more actions may include allowing or denying theproposed dialog act, based on the impact. For example, if the impact isbelow a threshold, the proposed dialog act may be entered at the currentpoint in the real-time conversation. In another example, if the impactis above a threshold, the proposed dialog act may not be entered at thecurrent point in the real-time conversation.

Further, in one embodiment, the one or more actions may include loggingthe impact (e.g., within a database, in association with a log of thereal-time conversation, etc.). In another embodiment, a plurality ofdifferent objectives of the first participant (or other participantsdifferent from the first participant) may be received for the real-timeconversation between the first participant and the second participant,and an impact of the proposed dialog act on each objective may bedetermined.

In this way, an impact of proposed dialog acts may be determined in anobjective-oriented manner, utilizing a constructed analytical model. Byanalyzing proposed dialog acts based on prior conversation turns with agoal of achieving an optimal implementation of a specifically statedobjective, this implementation may improve upon earlier predictionimplementations, which may only perform basic recognition and labelingactions/analysis. Additionally, analyzing and potentially filteringproposed dialog acts based on their impact may result in more efficientdialog acts being used within a conversation, which may minimize anecessary amount of turns within the conversation, and which may in turnreduce an amount of network traffic between the first participant andthe second participant as well as processing resources necessary toimplement such conversation (e.g., at a device of the first and/orsecond participant, at a network data transmission device, etc.).

Now referring to FIG. 6, a flowchart of a method 600 for training amodel to determine dialog acts for a conversation is shown according toone embodiment. The method 600 may be performed in accordance with thepresent invention in any of the environments depicted in FIGS. 1-3 and8-9, among others, in various embodiments. Of course, more or lessoperations than those specifically described in FIG. 6 may be includedin method 600, as would be understood by one of skill in the art uponreading the present descriptions.

Each of the steps of the method 600 may be performed by any suitablecomponent of the operating environment. For example, in variousembodiments, the method 600 may be partially or entirely performed byone or more servers, computers, or some other device having one or moreprocessors therein. The processor, e.g., processing circuit(s), chip(s),and/or module(s) implemented in hardware and/or software, and preferablyhaving at least one hardware component may be utilized in any device toperform one or more steps of the method 600. Illustrative processorsinclude, but are not limited to, a central processing unit (CPU), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), etc., combinations thereof, or any other suitablecomputing device known in the art.

As shown in FIG. 6, method 600 may initiate with operation 602, where aplurality of communications each having an associated objective areidentified. In one embodiment, the plurality of communications mayinclude a plurality of historical conversations. In another embodiment,the historical conversations may be identified as a result ofmonitoring. In yet another embodiment, each historical conversation mayhave an identified objective. In still another embodiment, eachhistorical conversation may be between a first and second participant,or between other participants.

Additionally, method 600 may proceed with operation 604, where aplurality of dialog acts are determined within the plurality ofcommunications. In one embodiment, one or more of the plurality ofdialog acts may be automatically identified (e.g., utilizing apre-trained dialog act classifier, etc.). In another embodiment, one ormore of the plurality of dialog acts may be manually identified.

Further, method 600 may proceed with operation 606, where a plurality offeatures are determined within each of the plurality of communications.Table 1 illustrates exemplary features that may be determined from theplurality of communications, in accordance with one embodiment. Ofcourse, it should be noted that the exemplary features shown in Table 1are set forth for illustrative purposes only, and thus should not beconstrued as limiting in any manner.

TABLE 1 An indication of a current turn in the dialog One or moreprevious dialog acts from opposite participant (e.g., customer in acustomer-agent conversation). Such dialog acts may be manually labeledor may be detected using a trained dialog act classifier. A previousdialog act from current participant (e.g. the agent) One or morepersonality attributes of both participants A topic of the dialogcomputed from previous conversations One or more tones exhibited inprevious turns An outcome of the conversation Word usage/Punctuation:binary bag-of-word unigrams, binary existence of a question mark, binaryexistence of an exclamation mark Temporal values: response time of aturn (time in seconds elapsed between the posting time of the previousturn and that of the current turn) Second-Person Reference values:existence of an explicit second-person reference in the turn (you, your,you're) Emotion values for participants: count of words in each of the 8emotion classes from the NRC emotion lexicon (anger, anticipation,disgust, fear, joy, negative, positive, sadness, surprise, and trust)Dialog values: lexical indicators for opening greetings (hi, hello,greetings, etc.), closing greetings (bye, goodbye), yes-no questions(turns with questions starting with do, did, can, could, etc.), wh-questions (turns with questions starting with who, what, where, etc.),thanking (thank*), apology (sorry, apolog*), yes-answer, and no-answer

Also, method 600 may proceed with operation 608, where a model istrained, utilizing the plurality of communications and associatedobjectives, the plurality of dialog acts, and the plurality of features.In one embodiment, the training may include associating each of theplurality of dialog acts with one or more features and an associatedobjective within the plurality of communications. In another embodiment,the model may include a sequential machine learning model (e.g., aMarkov model, etc.). In yet another embodiment, the model may determinea dialog act to be entered that maximizes an associated objective basedon the communications and plurality of features.

Now referring to FIG. 7, a flowchart of a method 700 for training amodel to score proposed dialog acts for a conversation is shownaccording to one embodiment. The method 700 may be performed inaccordance with the present invention in any of the environmentsdepicted in FIGS. 1-3 and 8-9, among others, in various embodiments. Ofcourse, more or less operations than those specifically described inFIG. 7 may be included in method 700, as would be understood by one ofskill in the art upon reading the present descriptions.

Each of the steps of the method 700 may be performed by any suitablecomponent of the operating environment. For example, in variousembodiments, the method 700 may be partially or entirely performed byone or more servers, computers, or some other device having one or moreprocessors therein. The processor, e.g., processing circuit(s), chip(s),and/or module(s) implemented in hardware and/or software, and preferablyhaving at least one hardware component may be utilized in any device toperform one or more steps of the method 700. Illustrative processorsinclude, but are not limited to, a central processing unit (CPU), anapplication specific integrated circuit (ASIC), a field programmablegate array (FPGA), etc., combinations thereof, or any other suitablecomputing device known in the art.

As shown in FIG. 7, method 700 may initiate with operation 702, where aplurality of communications each having an associated objective areidentified. In one embodiment, the plurality of communications mayinclude a plurality of historical conversations. In another embodiment,the historical conversations may be identified as a result ofmonitoring. In yet another embodiment, each historical conversation mayhave an identified objective. In still another embodiment, eachhistorical conversation may be between a first and second participant,or between other participants.

Additionally, method 700 may proceed with operation 704, where aplurality of dialog acts are determined within the plurality ofcommunications. In one embodiment, one or more of the plurality ofdialog acts may be automatically identified (e.g., utilizing apre-trained dialog act classifier, etc.). In another embodiment, one ormore of the plurality of dialog acts may be manually identified.

Further, method 700 may proceed with operation 706, where a plurality offeatures are determined within each of the plurality of communications.Table 1 illustrates exemplary features that may be determined from theplurality of communications, in accordance with one embodiment.

Further still, method 700 may proceed with operation 708, where avalidity score associated with each of the plurality of dialog acts isdetermined. In one embodiment, the validity score for a dialog act mayinclude an impact of the dialog act on the objective of thecommunication. For example, the validity score may indicate whether theassociated dialog act helped achieve the objective of the communication,and to what degree. In another embodiment, the validity score may be anumerical value (e.g., a whole number on a scale of 1-10, etc.).

Also, method 700 may proceed with operation 710, where a model istrained, utilizing the plurality of communications and associatedobjectives, the plurality of dialog acts, the plurality of features, andthe validity score associated with each of the plurality of dialog acts.In one embodiment, the training may include associating the validityscore associated with each of the plurality of dialog acts with one ormore features and an associated objective within the plurality ofcommunications.

FIG. 8 illustrates an exemplary dialog act recommendation system 800,according to one embodiment. As shown, the system 800 includes a featureextraction module 802 that identifies features from inputs 816. Theinputs 816 include logged conversations 808 as well as a set of dialogacts 812 extracted from the logged conversations 808 by a dialog actannotation/detection module 810. In one embodiment, each dialog act inthe set of dialog acts 812 may be manually identified, or may bedetected for each utterance of the logged conversations 808 using apre-trained dialog act classifier.

The inputs 816 also include associated conversation outcomes 804. Forexample, the logged conversations 808 may include prior textualcommunications between different participants, and the associatedconversation outcomes 804 of each textual communication. For instance,one example of the logged conversations 808 may include a customercomplaint, and one example of the associated conversation outcome 804for that customer complaint may include customer satisfaction with aCSR's solution to their complaint.

Further, the feature extraction module 802 may extract one or morefeatures from the inputs 816. Table 1 illustrates exemplary featuresthat may be extracted from the plurality of inputs 816, in accordancewith one embodiment.

Further still, one or more features 820 extracted from the plurality ofinputs 816 by the feature extraction module 802 are sent to the nextbest dialog act model training module 818. One or more optimalityobjectives 814 associated with the plurality of inputs 816 are also sentto the training module 818. Utilizing the optimality objectives 814 andthe plurality of inputs 816, the training module 818 trains arecommendation engine 822. In one embodiment, the recommendation engine822 may utilize a machine learning model that is trained by the trainingmodule 818.

Also, the trained recommendation engine 822 receives textual data from areal-time conversation 824. The trained recommendation engine 822 mayalso receive an optimality objective for the real-time conversation 824.In response, the trained recommendation engine 822 may output arecommendation of a next best dialog act 828, which may include arecommended dialog act to be entered during the next turn in thereal-time conversation 824 in order of maximize a likelihood ofrealizing the optimality objective.

In one embodiment, the system 800 may be trained on dialog-act labeledconversations in a customer service scenario, and conversation outcomes(e.g. satisfaction, problem resolution). The conversation may be betweenhuman and human (agent), human or automated agent, or between twoautomated agents. Such conversation can be between two parties, ormultiple parties. Using such training examples, the system 800 may traina model that predicts next best response (i.e. dialog act).

Additionally, in one embodiment, machine learning models used by therecommendation engine 822 may set the optimality objective to optimizean outcome such as to maximize customer satisfaction, minimize customerfrustration, etc. Thus, the trained models may be used to recommend thenext dialog act. In another embodiment, such recommendation may combinedialog acts and tones. A human agent may choose the next dialog act fromthe recommendations, or ignore the recommendation if he/she wants to.The recommendation may be computed from training a machine learningmodel from example conversations labeled with dialog acts in each turn.For each such conversation, an outcome may also be labeled. This mayserve as the ground truth for training models. Once such a model istrained, for a given conversation flow (unlabelled) and for a given userin the conversation, the model may predict the next best dialog act.Such prediction can be useful to generate a recommendation to humanagents, or select appropriate action for an automated agent.

Further, in one embodiment, input to train the machine learning modelsused by the recommendation engine 822 may include a set of conversations808, where an outcome 804 of each of the conversations is identified.For each utterance of the conversations 808, a dialog act 812 is eithermanually identified or detected using a pretrained dialog actclassifier. In order to train the model, many such training examples(conversations with their dialog act identified and outcome) may begiven.

In the feature extraction module 802, a set of features are extractedthe training example inputs 816. Most features may be identified fromthe content of the conversations 808. However, for computingconversation participant's personality traits, their previousconversations, or text from social media or other sources, may becollected. In one embodiment, such traits may be computed from pre-takensurveys.

The training module 818 also takes an optimality objective 814 as aninput (e.g., maximizing customer satisfaction, etc.). From such trainingexample inputs 816, computed features 820, and the optimality objective814, a machine learning model used by the recommendation engine 822 maybe trained. In one embodiment, such a model may be a sequential SVM-HMMthat may capture sequential properties of turns in a conversation. Oncesuch a model is trained, it may be used to predict a next best dialogact 828 for an input conversation 824.

FIG. 9 illustrates an exemplary dialog act validation system 900,according to one embodiment. As shown, the system 900 includes a featureextraction module 902 that identifies features from inputs 916. Theinputs 916 include logged conversations 908 as well as a set of dialogacts 912 extracted from the logged conversations 908 by a dialog actannotation/detection module 910. In one embodiment, each dialog act inthe set of dialog acts 912 may be manually identified, or may bedetected for each utterance of the logged conversations 908 using apre-trained dialog act classifier.

The inputs 916 also include associated conversation outcomes 904. Forexample, the logged conversations 908 may include prior textualcommunications between different participants, and the associatedconversation outcomes 904 of each textual communication. For instance,one example of the logged conversations 908 may include a customercomplaint, and one example of the associated conversation outcome 904for that customer complaint may include customer satisfaction with aCSR's solution to their complaint.

Additionally, the inputs 916 further include dialog acts with associatedvalidity scores 906. For example, the dialog acts with associatedvalidity scores 906 may include dialog acts found within the loggedconversations 908 that have predetermined validity scores assigned tothem. In one embodiment, each of the predetermined validity scores mayindicate an effectiveness of the associated dialog act in achieving oneor more predetermined optimality objectives.

Further, the feature extraction module 902 may extract one or morefeatures from the inputs 916. Table 1 illustrates exemplary featuresthat may be extracted from the plurality of inputs 916, in accordancewith one embodiment.

Further still, one or more features 920 extracted from the plurality ofinputs 916 by the feature extraction module 902 are sent to the dialogact validation model training module 918. One or more optimalityobjectives 914 associated with the plurality of inputs 916 are also sentto the training module 918. Utilizing the optimality objectives 914 andthe plurality of inputs 916, the training module 918 trains a validationengine 922.

Also, the trained validation engine 922 receives a proposed dialog act926 and a real-time conversation 924 in which the proposed dialog act926 is to be entered. The trained validation engine 922 may also receivean optimality objective for the real-time conversation 924. In response,the trained validation engine 922 may output a validation score 928 forthe proposed dialog act 926, which may indicate a likelihood that theproposed dialog act 926 achieves the optimality objective.

Table 2 illustrates an exemplary conversation between a customer and anagent, as well as identified dialog acts, in accordance with oneembodiment. Of course, it should be noted that the exemplaryconversation shown in Table 2 is set forth for illustrative purposesonly, and thus should not be construed as limiting in any manner.

TABLE 2 TURN # SPEAKER MESSAGE TEXT DIALOG ACTS 1 Customer Love my newSMASHED box for my Complaint, Negative product! Expressive Statement,Sarcasm 2 CSR That's disappointing! Truly sorry. Is the Request Info,Yes/No actual product damaged? Question, Apology 3 Customer No, however,I am a collector and I keep Negative Answer, the product in the box.Informative Statement 4 CSR I understand. Would you like theAcknowledgement, product to be exchanged? Yes/No Question, Offer 5Customer If possible, yes. I've never made a return Affirmative Answer,though. Informative Statement 6 CSR How did you purchase the product?Did Open Question, Request your local store assist with the order? Info,Yes/No Question 7 Customer I purchased the product online. As far asInformative Statement I know this came straight to my house through thewebsite.

Dialog Act Validation Example

In one embodiment, the conversation in Table 2 may be a real-timeconversation. In another embodiment, the objective of the conversationfor the CSR in Table 2 may be identified as “customer satisfaction.” Forthis conversation, the CSR may submit a message to be entered as Turn 8of the conversation (e.g., in response to Turn 7 of the conversation).In response, the text of the submitted message may be analyzed by thesystem, and one or more dialog acts may be derived by the system fromthe submitted message.

For example, if the CSR inputs the message text “please visit thefollowing URL to initiate a return through the website,” the system mayidentify a “suggestion” dialog act within the message text. In anotherexample, if the CSR inputs the message text “can you provide me with anorder number for your product order?”, the system may identify a“request information” dialog act within the message text. In yet anotherexample, if the CSR inputs the message text “we cannot handle websiteorder returns”, the system may identify a “negative expression” dialogact within the message text.

Additionally, in one embodiment, each of the one or more dialog acts maybe analyzed utilizing a validation model to determine a validity scorefor each dialog act, where the validation model is trained utilizinghistorical conversations involving the CSR and/or the customer. Forexample, the “suggestion” dialog act may be analyzed in view of the“customer satisfaction” objective utilizing the validation model, andmay be given a validity score of 10 (on a scale of 1-10, indicating ahigh likelihood that the objective will be obtained by the CSR inresponse to the message).

In another example, the “request information” dialog act may be analyzedin view of the “customer satisfaction” objective utilizing thevalidation model, and may be given a validity score of 5 (on a scale of1-10, indicating a medium likelihood that the objective will be obtainedby the CSR in response to the message). In yet another example, the“negative expression” dialog act may be analyzed in view of the“customer satisfaction” objective utilizing the validation model, andmay be given a validity score of 1 (on a scale of 1-10, indicating a lowlikelihood that the objective will be obtained by the CSR in response tothe message).

Further, in one embodiment, a second objective of the conversation maybe submitted (e.g., “problem resolution,” etc.), and each of the one ormore dialog acts may be analyzed utilizing a validation model todetermine a second validity score for each dialog act. In anotherembodiment, each objective of the conversation may be presented to theCSR, along with validity scores associated with that objective. In yetanother embodiment, the message to be entered may be automaticallyentered as part of the conversation (e.g., as Turn 8 of theconversation) if a validity score associated with all dialog acts forthe message are above a predetermined threshold. In still anotherembodiment, the message to be entered may be denied if a validity scoreassociated with one or more dialog acts for the message are below apredetermined threshold.

Further still, in one embodiment, the conversation in Table 2 may be ahistorical conversation used to train the validation engine. Forexample, dialog acts and associated validity scores may be manually ordynamically identified for message text in each turn of theconversation. As a result, for a given conversation and expected outcomeof the conversation, if there are N pairs of proposed dialog acts andassociated validity scores given as training, this may correspond to Ntraining examples.

In order to train the model, many such training examples from one ormore conversations may be given. During feature extraction, a set offeatures may be extracted from such training examples. In oneembodiment, features may be identified from the content of theconversations. In another embodiment, for computing conversationparticipant's personality traits, their previous conversations, or textfrom social media or other sources, may be collected.

For example, such traits may be computed from pre-taken surveys. Thetraining method may also require an optimality objective as an input(e.g., “maximizing customer satisfaction,” etc.). Such trainingexamples, in addition to features computed from the examples, dialogacts and associated validity scores, and optimality objectives may allbe used to train a machine learning model. In one embodiment, the modelmay be a sequential SVM-HMM that captures sequential properties of turnsin a conversation. Once such a model is trained, it may be used topredict a validity score of a proposed dialog act for a givenconversation.

Dialog Act Recommendation Example

In one embodiment, the conversation in Table 2 may be analyzed,utilizing a trained model. For example, one or more features may beextracted from the conversation, as well as the dialog acts. Anobjective of the conversation may also be identified (e.g., by the CSRor other user of the system). The features, dialog acts, and objectivemay be compared to features, dialog acts, and associated objectives fromprevious conversations used to train the model. Based on thiscomparison, one or more dialog acts may be recommended for entry as partof the conversation (e.g., as Turn 8 of the conversation).

For example, based on the analysis, the system may determine that a“suggestion” dialog act maximizes a likelihood of obtaining theobjective of the conversation. This recommended dialog act may then bepresented to the CSR. In another embodiment, a sequence of dialog actsmay be recommended for entry as part of the conversation (e.g., asmultiple future turns of the conversation). For example, the system maydetermine that a “suggestion” dialog act, followed by a “requestinformation” dialog act during the CSR's next turn of the conversation,maximizes a likelihood of obtaining the objective of the conversation.These recommended dialog acts, as well as their associated order, maythen be presented to the CSR.

Additionally, in one embodiment, after presenting the one or morerecommended dialog acts, the system may receive a message to be entered(e.g., as Turn 8 of the conversation). This message may be analyzed inorder to determine one or more dialog acts performed by the message. Inanother embodiment, the message to be entered may be automaticallyentered as part of the conversation (e.g., as Turn 8 of theconversation) if the one or more dialog acts performed by the messagematch the one or more recommended dialog acts. In still anotherembodiment, the message to be entered may be denied if the one or moredialog acts performed by the message do not match the one or morerecommended dialog acts.

Further, in one embodiment, a second objective of the conversation maybe submitted (e.g., “problem resolution,” etc.), and additionalrecommended dialog acts may be identified for the second objective. Inanother embodiment, each objective of the conversation may be presentedto the CSR, along with recommended dialog acts associated with thatobjective.

Further still, in one embodiment, the conversation in Table 2 may bebetween a customer and an agent, and the recommendation may be made toan agent. In another embodiment, such recommendation may be made to acustomer. In another embodiment, a next best sequence of dialog acts maybe predicted to achieve the target objective. In such a case, therecommendation engine may recommend a sequence of dialog acts instead ofa single next dialog act.

In this way, a model may be built to recommend dialog acts to a specificparty in a conversation. This recommendation may be computed from priorconversation turns, and with a goal to achieve an optimum outcome. Sucha system may be trained from a set of conversations with an outcome(e.g., customer satisfaction) and features identified from theconversations. One or more of the following features may be included,such as n-grams from conversation turns, dialog acts from currentparticipant, and previous participant, temporal and topic basedfeatures, tone/emotion based features. A machine learning model may setits optimality objective to optimize an outcome such as to maximizecustomer satisfaction, minimize customer frustration, etc. Once such amodel is trained, for a given conversation flow, and for a given user inthe conversation, the model may recommend the next best dialog act(e.g., apology, etc.) to optimize the target outcome (e.g., maximizecustomer satisfaction). The system may be used as a method to furtherautomation of the customer service loop, and for automaticresponse-generation systems.

In one embodiment, a validity score may be given to a dialog actprovided by an agent (human or automated) in a customer service dialog.Given some defined desired outcome of a customer service conversation,for example, customer satisfaction, the system may judge the dialog actprovided by the agent based on how it will change the probability of thedesired outcome (e.g., overall customer satisfaction, etc.) at thecurrent point (i.e. current turn) in the conversation. The dialog act tobe judged may be human-generated or automatically generated, and mayrequire ranking for best-dialog act selection based on full conversationoutcome prediction. In another embodiment, dialog acts for eachconversation turn may be manually annotated. In another embodiment, adialog act detector may be used to detect such acts for eachconversation turn.

Additionally, in one embodiment, in order to judge a dialog act, thesystem may use a model which takes the current response with its dialogact identified (or annotated), and responses by each party from previousconversation turns (e.g., both customer and agent response), andcomputes a validity score for the current dialog act which indicates thelikelihood of the target outcome (e.g., customer satisfaction, etc.).Such a system may be trained from a set of conversations with an outcome(e.g., customer satisfaction, etc.) and features identified from theconversations. Features may include n-grams from conversation turns,dialog acts from current participant, and previous participant, temporaland topic based features, and tone/emotion based features. A machinelearning model may set an optimality objective to optimize an outcomesuch as to maximize customer satisfaction, minimize customerfrustration, etc.

Further, in on embodiment, once such a model is trained, the model maypredict, for a given conversation flow, a given user in theconversation, and the current dialog act from that user, a validityscore (i.e. likelihood score) to achieve a target outcome (e.g.,customer satisfaction). In this way, the system may provide real-timevalidation to agents about whether their intended response will increasethe chances of a desired conversational outcome.

Further still, the system may be used as both a method to furtherautomation of the customer service loop, and as a training method forproviding feedback for agents and automatic response-generation systems.

In one embodiment, the recommendation engine may be modeled as asequential decision making problem, where the notion of sequence comesfrom how a conversation unfolds utterance by utterance in a sequentialmanner. At each stage of a sequence (e.g., each utterance, etc.), thesystem may determine a dialog act to recommend that maximizes theoverall outcome (e.g. satisfaction, problem resolution) of theconversation.

In another embodiment, reinforcement learning (RL) may be used to solvesequential decision making problems. The learning entity here mayinclude the recommendation engine and it may be equipped with a rewardfunction (derived from the conversation outcome) that governs how theengine is doing in terms of recommendation quality. For example, thereward function may give the engine a positive reward for arecommendation of a dialog act indicating a successful outcome of aconversation, and may give a negative reward to the engine otherwise.

Such reward system may motivate the learning entity to select the bestrecommendation in order to maximize rewards. Markov decision process(MDP) is one RL technique that may be used to implement therecommendation engine.

Markov Decision Process

A Markov decision process (MDP) may be represented as a tuple of (S, A,P_(sa), [ ], R), where:

-   -   S=Set of states. In one example, a state may be any possible        sequence of dialog acts, for example, (complaint, request for        information, give information) is a state. States may also        incorporate how many utterances have passed in a conversation        (in this example, 3 utterances).    -   A=Set of actions. In the above problem setup, actions may be        mapped to recommendations (e.g., a set of dialog acts).    -   P_(sa)=State transition probabilities. This may represent a        distribution of what states the system will be in after        executing action a in state s.    -   [ ]=Discount rate (the value may be between 0 and 1).    -   R=Reward function defined as R:SXA→        . It may measure what real valued reward the system will get        once it performs action a in state s.

FIG. 10 illustrates an exemplary progression 1000 of an MDP over time.As shown, a process starts from state s₀ 1002 then after action a₀ 1004it moves to state Si 1006, where s₁ 1006 is drawn from transitionprobability distribution P_(s0, a0). Similar operations occur in statess₂ 1008, s₃ 1010, and so on.

In a dialog act recommendation setup, in each state a dialog act isrecommended before moving to the next state. After visiting all thestates s₀ 1002, s₁ 1006, etc. with actions a₀ 1004, a₁ 1012, etc., thetotal payoff may be computed as [R(s₀, a₀)+R(s₁, a₁)+2 R(s₂, a₂)+ . . .]. One goal of MDP may be to select actions in each state in such a wayso that expected value E[R(s₀, a₀)+R(s₁, a₁)+2 R(s₂, a₂)+ . . . ] ismaximized. Finally, the output of a MDP process may be known as a policywhich may include a set of actions taken in each state of the process.In one embodiment, an MDP-based recommendation engine may recommend abest dialog act at each stage of a conversation.

MDP-Based Dialogue Acts Recommendation Engine

In one embodiment, three dialogs acts may exist {Complaint, Question andAnswer} and a maximum length of a conversation may be 3. Each of themodules (e.g., states, actions, transition probabilities, reward, andpolicy) of an MDP may now be discussed with respect to this problemsetup. Note that, we will be using parenthesis and curly-braces toindicate state and action, respectively.

State: A state may be any possible sequence of dialog acts. In ourexample, (Complaint), (Complaint, Question) or (Question, Question) areinstances of valid states. A length of a state may correspond to thelength of a conversation so far. In one embodiment, the state transitionmay only move forward (e.g., from 1 length to 2 length to 3 length,etc.). This property may be valid since, in a conversation, we alwaysmove forward from utterance 1 to utterance 2 and so on.

Action: The set of dialog acts may correspond to the set of actions. Inthis example, {Complaint, Question, Answer} is a set of possible actionsfor each state.

State Transition Probabilities: As discussed above, a state transitionprobability may correspond to a possibility of moving from one state toanother while executing an action. According to one example, fortransition from a (Complaint) state to a (Complaint, Question) statewith an action {Question}, we need to compute P_((complaint){Question})((Complaint, Question)). We may compute this probability from data usingthe following equation shown in Table 3, in accordance with oneembodiment. Of course, it should be noted that the exemplary equationshown in Table 3 is set forth for illustrative purposes only, and thusshould not be construed as limiting in any manner.

TABLE 3 $\quad\begin{matrix}{{P_{{({complaint})}{({Question})}}\mspace{14mu} ( ( {{Complaint},{Question}} ) )} =} \\\frac{{Frequency}( ( {{Complaint},{Question}} ) )}{{Frequency}\mspace{14mu} ( ({Complaint}) )}\end{matrix}$

Reward: The reward function may quantify the choice of an action in astate. In our case, the engine may pick such action (e.g., dialog act,etc.) for recommendation that has a higher potential to make theconversation successful (e.g. maximizing the determined objective suchas satisfaction, problem resolution, etc.). For example, we can computethe reward of recommending {Question} in state (Complaint) with anobjective of making the conversation satisfied, resolute from data usingthe following equation shown in Table 4, in accordance with oneembodiment. Of course, it should be noted that the exemplary equationshown in Table 4 is set forth for illustrative purposes only, and thusshould not be construed as limiting in any manner.

TABLE 4$\quad{{R( ( {{Complaint},{Question}} ) )} = {( \frac{\begin{matrix}{\# \mspace{14mu} {of}\mspace{11mu} {satified}\mspace{14mu} {conversations}\mspace{14mu} {with}} \\{({Question})\mspace{14mu} {followed}\mspace{14mu} {by}\mspace{11mu} ({Complaint})}\end{matrix}}{\begin{matrix}{\# \mspace{14mu} {of}\mspace{11mu} {total}\mspace{14mu} {conversations}\mspace{14mu} {with}} \\{({Question})\mspace{14mu} {followed}\mspace{14mu} {by}\mspace{11mu} ({Complaint})}\end{matrix}} ) +}}$ $( \frac{\begin{matrix}{\# \mspace{14mu} {of}\mspace{11mu} {resolvved}\mspace{14mu} {conversations}\mspace{14mu} {with}} \\{({Question})\mspace{14mu} {followed}\mspace{14mu} {by}\mspace{11mu} ({Complaint})}\end{matrix}}{\begin{matrix}{\# \mspace{14mu} {of}\mspace{11mu} {total}\mspace{14mu} {conversations}\mspace{14mu} {with}} \\{({Question})\mspace{14mu} {followed}\mspace{14mu} {by}\mspace{11mu} ({Complaint})}\end{matrix}} )$

Policy: The policy may include a set of optimal recommendations indifferent states.

Note that, in the above discussion an assumption may be made that theengine will make one dialog act recommendation in each state, but it isalso possible to extend the definition states and make multiple dialogact recommendation in each state.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein includes anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which includes one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Moreover, a system according to various embodiments may include aprocessor and logic integrated with and/or executable by the processor,the logic being configured to perform one or more of the process stepsrecited herein. By integrated with, what is meant is that the processorhas logic embedded therewith as hardware logic, such as an applicationspecific integrated circuit (ASIC), a FPGA, etc. By executable by theprocessor, what is meant is that the logic is hardware logic; softwarelogic such as firmware, part of an operating system, part of anapplication program; etc., or some combination of hardware and softwarelogic that is accessible by the processor and configured to cause theprocessor to perform some functionality upon execution by the processor.Software logic may be stored on local and/or remote memory of any memorytype, as known in the art. Any processor known in the art may be used,such as a software processor module and/or a hardware processor such asan ASIC, a FPGA, a central processing unit (CPU), an integrated circuit(IC), a graphics processing unit (GPU), etc.

It will be clear that the various features of the foregoing systemsand/or methodologies may be combined in any way, creating a plurality ofcombinations from the descriptions presented above.

It will be further appreciated that embodiments of the present inventionmay be provided in the form of a service deployed on behalf of acustomer to offer service on demand.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred embodiment shouldnot be limited by any of the above-described exemplary embodiments, butshould be defined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A computer-implemented method, comprising:receiving, utilizing a processor, textual data associated with areal-time conversation between a first participant and a secondparticipant; receiving, utilizing the processor, an objective of thefirst participant for the real-time conversation between the firstparticipant and the second participant; receiving, utilizing theprocessor, a proposed dialog act to be entered at a current point in thereal-time conversation from the first participant; determining,utilizing the processor, an impact of the proposed dialog act on theobjective of the first participant, utilizing a model; and performing,utilizing the processor, one or more actions based on the impact of theproposed dialog act.
 2. The computer-implemented method of claim 1,wherein the objective includes textual data indicating a desired outcomeof the real-time conversation for the first participant.
 3. Thecomputer-implemented method of claim 1, wherein the objective includes amaximization of satisfaction of the second participant with respect tothe real-time conversation, a minimization of frustration by the secondparticipant with respect to the real-time conversation, a resolution ofa problem of the second participant as a result of the real-timeconversation, or an answering of a question by the first participantduring the real-time conversation.
 4. The computer-implemented method ofclaim 1, wherein the proposed dialog act includes a description of anaction that is to be performed by a sending of one or more portions ofthe textual data from one participant to another participant within thereal-time conversation.
 5. The computer-implemented method of claim 1,wherein the proposed dialog act is identified from textual inputprovided by the first participant.
 6. The computer-implemented method ofclaim 1, wherein determining the impact includes scoring the proposeddialog act on a predetermined scale.
 7. The computer-implemented methodof claim 1, wherein determining the impact includes determining anamount that the proposed dialog act will change a probability of theobjective of the first participant being achieved during the currentpoint in the real-time conversation.
 8. The computer-implemented methodof claim 1, wherein determining the impact includes applying theproposed dialog act to the model.
 9. The computer-implemented method ofclaim 8, wherein the model is trained by identifying dialog acts,features, and objectives of earlier conversations.
 10. Thecomputer-implemented method of claim 9, wherein the features areselected from a group consisting of: an indication of a current turn ina dialog, one or more previous dialog acts from the second participant,a previous dialog act from the first participant, one or morepersonality attributes of both the first participant and the secondparticipant, a topic of a dialog computed from previous conversations,one or more tones exhibited in previous turns, an outcome of aconversation, word usage and punctuation, temporal values, second personreference values, emotion values, and dialog values.